The Benefits of Learning with Strongly Convex Approximate Inference *** Supplemental ***
نویسندگان
چکیده
Strong convexity can be characterized in a number of ways. The following facts provide some conditions that are equivalent to Definition 1. For the 2-norm, Fact 2 means that the minimum eigenvalue of the Hessian is lower-bounded by κ. Recall that˜µ(θ) and˜µ(θ) are the gradients of˜Φ(θ) and˜Φ(θ), respectively. Since the conjugate function, ˜ Φ * , is assumed to be κ-strongly convex, we have via Lemma 1 and Definition 2 that˜µ(θ) − ˜ µ(θ) 2 = ∇ ˜ Φ(θ) − ∇ ˜ Φ(θ) 2 ≤ 1 κ θ − θ 2. (14) Dividing both sides by |G| completes the proof. B.2. The Marginals of the Expected NLL Minimizer are the True Marginals Observe thatˆθ m effectively fits the empirical marginals of the dataset, 1 m m j=1ˆy (j). Thus, as m → ∞, the marginals induced byˆθ m and θ converge. This is formalized in the following lemma. Lemma 4. Let µ(θ) denote the true marginals of a distribution. Let ¯ θ denote the minimizer of the expected NLL, per Eq. 5. Then, µ(θ) = ˜ µ(¯ θ).
منابع مشابه
The Benefits of Learning with Strongly Convex Approximate Inference
We explore the benefits of strongly convex free energies in variational inference, providing both theoretical motivation and a new meta-algorithm. Using the duality between strong convexity and stability, we prove a high-probability bound on the error of learned marginals that is inversely proportional to the modulus of convexity of the free energy, thereby motivating free energies whose moduli...
متن کاملA Q-learning Based Continuous Tuning of Fuzzy Wall Tracking
A simple easy to implement algorithm is proposed to address wall tracking task of an autonomous robot. The robot should navigate in unknown environments, find the nearest wall, and track it solely based on locally sensed data. The proposed method benefits from coupling fuzzy logic and Q-learning to meet requirements of autonomous navigations. Fuzzy if-then rules provide a reliable decision maki...
متن کاملEvaluation of the Efficiency of the Adaptive Neuro Fuzzy Inference System (ANFIS) in the Modeling of the Ionosphere Total Electron Content Time Series Case Study: Tehran Permanent GPS Station
Global positioning system (GPS) measurements provide accurate and continuous 3-dimensional position, velocity and time data anywhere on or above the surface of the earth, anytime, and in all weather conditions. However, the predominant ranging error source for GPS signals is an ionospheric error. The ionosphere is the region of the atmosphere from about 60 km to more than 1500 km above the eart...
متن کاملOn the quadratic support of strongly convex functions
In this paper, we first introduce the notion of $c$-affine functions for $c> 0$. Then we deal with some properties of strongly convex functions in real inner product spaces by using a quadratic support function at each point which is $c$-affine. Moreover, a Hyers–-Ulam stability result for strongly convex functions is shown.
متن کامل